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Abstract 

We generalize the notion of the stopping redundancy in order to study the smallest size of a trapping set in Tanner graphs of 
linear block codes. In this context, we introduce the notion of the trapping redundancy of a code, which quantifies the relationship 
between the number of redundant rows in any parity-check matrix of a given code and the size of its smallest trapping set. Trapping 
sets with certain parameter sizes are known to cause error-floors in the performance curves of iterative belief propagation decoders, 
and it is therefore important to identify decoding matrices that avoid such sets. Bounds on the trapping redundancy are obtained 
using probabilistic and constructive methods, and the analysis covers both general and elementary trapping sets. Numerical values 
for these bounds are computed for the [2640, 1320] Margulis code and the class of projective geometry codes, and compared with 
some new code-specific trapping set size estimates. 

Index Terms Belief Propagation, LDPC Codes, Margulis Codes, Projective Geometty Codes, Trapping Redundancy, Trapping 
Sets. 

I. Introduction 

The performance of linear error-correcting codes (and low-density parity-check (LDPC) codes in particular) under iterative 
decoding depends on the choice of the parity-check matrix of the code. More precisely, the error rate of a code is influenced by 
a class of combinatorial entities determined by the choice of the parity-check matrix, such as stopping [1] and trapping sets [2], 
[3]. Stopping and napping sets are defined in terms of consttaints on the weights of rows in the parity-check matrix induced by 
subsets of its columns. Certain such restrictions on the weight distributions of the rows can only be satisfied if the parity-check 
matrix of the code has a sufficiently large number of judiciously chosen rows. Thus, recent work focused on inttoducing redundant 
rows into parity-check matrices of a code in order to ensure that the size of their smallest stopping sets are sufficiently large or 
equal to the minimum distance of the code [4], [5], [6], [7]. Since adding redundant rows to the parity-check matrix increases the 
decoding complexity of the code, it is important to understand the inherent nade-off between the size of the smallest stopping set 
and the number of rows in a parity-check matrix. Several ideas for addressing these issues that exploit properties of orthogonal 
arrays and covering arrays [8] were described in [6], [4], and [9]. 

The contributions of this work are three-fold. First, we generalize the notion of the stopping redundancy for the case of napping 
sets, and term this combinatorial number the trapping redundancy. Second, we describe simple probabilistic and deterministic 
methods for upper-bounding the trapping redundancy of binary linear block codes. Third, we present new analytical techniques 
for estimating the sizes of small napping sets in the family of projective geometry (PG) codes and the Margulis [2640, 1320] 
code, and compare these estimates with the upper bounds. 

The paper is organized as follows. Section [II] provides relevant definitions and innoduces the terminology used throughout the 
paper. Section UTU contains the main results - probabilistic and constructive upper bounds on the trapping redundancy of codes. 
Section [IV] describes the relationship between trapping sets and arcs in PG codes. In the same section, numerical results for the 
trapping redundancy of the Margulis [2640, 1320] and the family of PG codes are compared with results concerning arcs and 
elementary trapping sets in the family of Margulis codes. Concluding remarks are given in Section [V] 

II. Definitions, Background, and Terminology 

We start by introducing napping sets and elementary napping sets, and then proceed to define the notions of the restriction of 
(redundant) parity-check matrices. Based on the notion of the restriction, we state the cennal definition of the paper, pertaining 
to the trapping redundancy of a linear block code. 
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Academic Exchange Service (DAAD) fellowship awarded to Thorsten Hehn. 



A. Near-Codewords and Trapping Sets 

Decoding of LDPC codes is usually performed in an iterative manner, using the suboptimal belief -propagation (BP) algorithm. 
This decoding approach, also known as message passing, can be seen as the process of exchanging reliability messages along 
the edges of a bipartite Tanner graph. The incidence matrix of the Tanner graph corresponds to the parity-check matrix used 
for decoding, so that the code variables index the vertices on the left hand side of the graph, while the parity-check equations 
index the right hand side vertices of the graph. For a more comprehensive treatment of LDPC codes and iterative decoding, the 
interested reader is referred to [10], [11], [12]. 

The error- floor phenomenon of iterative decoders was first described by MacKay and Postol in [2], who observed that the bit 
error rate curve of the [2640, 1320] Margulis code exhibits a sudden change of slope at signal-to-noise ratios approximately equal 
to 2.4 dB. This change of slope was attributed to the existence of near-codewords in the Tanner graph of the Margulis code with 
a parity-check matrix H described in [13]. Near codewords are error vectors y of small weight, with syndromes s y = Hy that 
also have small weight. In his seminal paper [3], Richardson analyzed the effect of near-codewords on the performance of various 
classes of decoders and for a group of channels. He also introduced the notion of trapping sets to describe configurations of variable 
nodes in Tanner graphs of codes that cause failures of specific decoding schemes. There exist many different groups of trapping 
sets. For example, trapping sets of maximum likelihood decoders are sets of variables containing the supports of codewords of 
the code; trapping sets of iterative decoders used for messages transmitted over the binary erasure channel are stopping sets [1]. 
For the additive white Gaussian noise (AWGN) channel and BP decoding, no simple characterization of trapping sets is known. 
Nevertheless, extensive computer simulations revealed that a large number of trapping sets for this channel/decoder combination 
can be described in a simple and precise setting. Henceforth, we use the notion "trapping set" to refer exclusively to sets of the 
form described below. To define trapping sets, we first introduce the notion of the restriction of a matrix. 

Definition 2.1: For a given mxn matrix H = (Hij) with 1 < i < m, 1 < j < n, the restriction of a set of t columns indexed 
by ■ ■ ■ ,jt is defined as an m x t sub-matrix of H consisting of the elements Hij, 1 < i < m, j = jx,32, ■ ■ ■ ,jt- 

For a given linear [n, k, d] code C, with parity-check matrix H and corresponding Tanner graph Q(H), trapping sets are defined 
as follows. 

Definition 2.2: An (a, b) trapping set T(a, b) is a collection of a variable nodes for which the subgraph in G{H) induced by 
T(a, b) and its neighbors contains b > odd-degree check nodefl Equivalently, an (a, b) trapping set T(a, b) of H is a set of 
a columns with a restriction that contains b odd-weight rows. 

The class of trapping sets that exhibits the strongest influence on the performance of iterative decoders is the class of elementary 
trapping sets. 

Definition 2.3: An elementary (a, b) trapping set is a set T' e )(a, b) for which all check nodes in the subgraph induced by 
T^ e \a,b) and its neighbors have either degree one or two, and there are exactly b degree-one check nodes. Alternatively, an 
elementary (a, b) trapping set is a trapping set for which all non-zero rows in the restriction have either weight one or two, and 
exactly b rows have weight one. 

For a fixed value of the parameter a (or 6), the problem of finding the trapping set with smallest parameter b (or a) in a given 
parity-check matrix is NP-hard, and NP-hard to approximate [14]. This makes a general and complete characterization of the 
trapping set sizes and trapping redundancy prohibitively complex. We therefore focus our attention on deriving upper bounds on 
the trapping redundancy with set sizes restricted to a < d — 1 only, where d denotes the minimum distance of the code, and in 
particular, elementary trapping sets [15]. This is motivated by the recent studies that suggest that trapping sets most detrimental 
to the code performance are elementary, and that they have small (a, b) parameters, usually such that b < a < d [3], [16]. 

B. Redundant Parity-Check Matrices 

For every linear [n, k, d] code C, there exist many choices for parity-check matrices, although for iterative decoding not all of 
them may be adequate. This is due to the fact that some parity-check matrices have irregular row- and column-weights and that 
they contain a large number of stopping and trapping sets [6]. In order to mitigate this problem, one may resort to the use of 
redundant parity-check matrices, i.e., matrices that contain more than n — k rows, although they have row-rank equal to n — k. 
Examples of the use of redundant parity-check matrices for signaling over the binary erasure channel can be found in [6], [17], 
[18]. 

Henceforth, we use the phrase redundant parity-check matrix to refer to a parity-check matrix with row-rank n — k that has more 
than n — k rows. A redundant parity-check matrix contains rows that are linear combination of other rows that represent a basis of 
the dual code C . For a fixed basis, rows of this form are referred to as redundant rows. On the other hand, a parity-check matrix 
of full row-rank and dimension (n — k) x n is simply termed a parity-check matrix. Redundant parity-check matrices are used 
to impose specific constraints on the structure of their corresponding Tanner graphs. As was shown in [19], even one judiciously 
chosen redundant row can be used to lower the error floor of the Margulis code. This is achieved in terms of rendering the structure 
of a selected small trapping set so as to increase the number of its corresponding unsatisfied parity-check equations. From an 
application point of view, it is of interest to identify one or a few redundant rows that can be added to the parity-check matrix 

'The case b = corresponds to codewords. Henceforth, we consider the case b > only. 
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in order to eliminate a trapping set causing a special instant of decoding failure. Note that one or a few redundant rows of low 
weight do not significantly alter the performance of a code in the waterfall region, although they may have a significant bearing 
on its performance in the error-floor region. In what follows, we consider the more general theoretical problem of determining 
the smallest number of redundant rows needed to simultaneously eliminate the negative effect of classes of trapping sets on the 
performance of iterative decoders. In this context, our results can be seen as a generalization of the findings in [6] for the case 
of trapping sets. 

An analytical study of the trapping redundancy is presented in the following section. 

III. The Trapping Redundancy: A Probabilistic Approach 

We investigate the fundamental theoretical trade-offs between the number of rows in a redundant parity-check matrix of a code 
and the size of its smallest trapping set with a given set of parameters. 

A. Definition and Bounds of the Trapping Redundancy 

In all our subsequent derivations, we make use of the following definition. 

Definition 3.1: ([8, p. 5]) An orthogonal array A of strength t is an array of dimensions to x n such that every mxt subarray 
contains each possible t-tuple as rows the same number of times. 

The codewords of an [n, k, d) linear code C form an orthogonal array of dimension 2 fe x n and strength d 1 - — 1, where d- 1 
denotes the dual distance of C. Note that if A is an array of strength t, then A is also an orthogonal array of strength s, for all 
integers s < t. 

Let 6h(cl, b) denote the number of (a, b) trapping sets in the parity-check matrix H. We have the following result for 9h{(i, b) 
corresponding to a matrix H that consists of all codewords of the dual code. 

Proposition 3.2: Let H consist of all 2"~ fc codewords of the dual code of an [n, k, d] linear code C, for n — k > 1. Then 
e H (a, b) = Q for all pairs (a, b) such that l<a<d-\, and b ^ 2"- fc - 1 . 

Proposition 13.21 shows that a parity-check matrix that consists of all codewords of the dual code cannot contain trapping sets 
with 1 < a < d — 1 variables with less than or more than 2"~ fc ~ 1 checks connected to them an odd number of times. This is 
a direct consequence of the fact that H in this case represents an orthogonal array, so that each restriction of 1 < a < d — 1 
columns of H contains each vector of length a the same number of times. Consequently, there are 2"~ fc ~ 1 rows in the restriction 
of the a columns that have even weight and 2"~ fc ~ 1 rows that have odd weight. 

However, it is of much larger importance to determine if there exist parity-check matrices with a number of rows significantly 
smaller than 2™~ fc that are free of trapping sets with fixed parameters (a, s), for all 1 < s < b. For this purpose, we introduce 
the notion of the (a, b) trapping redundancy of a code. 

Definition 3.3: The (a, b) trapping redundancy T a ,b(C) of an [n, k, d] linear code C is the smallest number of rows m of any 
(redundant) parity-check matrix which does not contain trapping sets with parameters (a, s), 1 < s < b. Similarly, the smallest 
number of rows T^ e ) (C) in a (redundant) parity-check matrix of C avoiding (a, s) elementary trapping sets with 1 < s < b is 
referred to as the (a, b) elementary trapping redundancy of C. 

Theorem 3.4: Let C be an [n, k,d) code and C its dual. Use Mc{m) to denote the ensemble of all to x n matrices with 
rows chosen independently and at random, with replacement, from the set of 2 n ~ k codewords of C . Furthermore, let 1 < a < 
[(d — 1)/2J be fixed, let 8(a, b) be the number of trapping sets with parameters (a, s), < s < b, b < m, in a randomly chosen 
matrix from A^c(w), and let e denote the base of the natural logarithm. If 




then P{<d(a,b) = 0} > 0. Consequently, if to satisfies ((TJ, then there exists a parity-check matrix of C with not more than 
to + n — k — 1 rows that does not contain any (a, s) trapping sets with 1 < s < b. 

Note that m + n — k — 1, for any to satisfying (Q~|i, represents an upper bound on the trapping redundancy of the code, i.e. 

T a ,b(C) < m + n — k — 1. 

Proof: The proof of the claimed result is based on Lovasz Local Lemma (LLL), stated below following the exposition 
of [20]. 

Lemma 3.5: Let E\, E 2 , ■ ■ ■ , be a set of events in an arbitrary probability space. Suppose that each event is independent 
of all other events Ej, 1 < i,j < N, except for at most r of them, and that P{Ei} < p for all 1 < i < N. If 

ep(r + l)<l, (2) 

*en P{n£Li E l }>0. 
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Let Ei be the event that the restriction of the i-th collection (out of (")) of a columns from a randomly chosen matrix in 
M.c(m) contains fewer than b odd rows. Then 



b-l 



2 

Equation (0) follows from the fact that the codewords of the dual code form an orthogonal array of strength d — 1, and that 
therefore even and odd weight rows in the restriction are equally likely. This is true independent of the choice of a, provided 
that 1 < a < [(d — 1 )/2j . The orthogonal array property of strength d — 1 implies that all row-vectors of length up to d — 1 are 
equally likely; henceforth, restricting the number of columns a to lie in the range 1 < a < [(d — 1)/2J ensures that two events 
Ei and Ej, i ^ j, are independent as long as their corresponding sets of column indices are disjoint. 

In the above setting, P{f] Ei} denotes the probability that the randomly chosen matrix is free of trapping sets with parameters 
(a, s), for all < s < b. In order to complete the proof, it suffices to observe that the following relationship holds for the 
dependence number r of the events Ei, 

Efa\ fn-a\ v^-> fa\ fn — a\ fn — a\ fn\ fn — a\ 

M (J(.-i) +1 -£(J(.-.H . )-(.)-( . )■ ,4) 

Expression (01 is a consequence of the fact that two collections of a columns are "dependent" if and only if they share at least 
one column. In other words, if one collection of a columns is fixed, another collection of the same size is deemed independent 
from it if its columns are chosen from the remaining set of n — a columns. The right-hand side of Equation (|4]i follows from the 
Vandermonde convolution formula, which asserts that 

( r \ I s \ I r + s 



^ \t + I J \u — I J \t + u 

where r, t, s, u denote non-negative integers. 

In the worst case, additional n — k — 1 rows may be needed to make the randomly chosen matrix have full row-rank n — k. 
This is due to the fact that a matrix containing no (a, 0) trapping sets must have rank at least one. 

Note that appending n — k — 1 additional rows to the selected set of m rows can only increase the number of odd-weight rows 
in each restriction, and hence cannot reduce the value of the parameter b. This completes the proof of the theorem. ■ 

Although Theorem l3.4l ensures the existence of at least one matrix in A4c(m) that is free of trapping sets with given parameters 
(a, b), the actual probability of selecting such a matrix may be very small. It is therefore of interest to identify values of the 
parameter m for which the probability of drawing a matrix of the desired form from the ensemble A^c(w) is close to one. 

Theorem 3.6: For a linear code C, let 0(a, b) be the number of trapping sets with parameters (a,s), 1 < a < [(d — 1)/2J, 
< s < b < m, that exist in an arbitrarily chosen m x n array from the M.c{ m ) ensemble. If 

(5) 

then P{<d(a, b) = 0} > 1 — e, where r is given by Equation and where < e < 1 is a real number. 

Consequently, if m satisfies (f5]), then for small values of e one can find with high probability a (redundant) parity-check matrix 
for C with not more than m + n — k — 1 rows that does not contain any (a, s) trapping sets with 1 < s < b. 

Proof: The result in Equation (O is obtained from the high-probability variation of LLL [20], stated below. 

Lemma 3.7: Let Ex,E2,... ,Ejsr be a set of events in an arbitrary probability space, and let < e < 1. Suppose that each 
event Ei is independent of all other events Ej, except for at most r of them. If 

P{Ei}< ±(l-±Y, l<t<N, (6) 




N V N 
then P{Cd Ei}>l-e. 

To prove the theorem, let Ei, 1 < i < N, denote the event that the i-th collection of a columns contains < s < b rows of 
odd weight. Replace the expression for P{Ei} in Equation © by the right-hand side of Equation <£3j and use the formula for r 
stated in Equation ■ 

We derive next upper bounds on the elementary trapping redundancy of linear block codes. 

Theorem 3.8: Let C be an [n, k, d] code and C its dual. Let Aic{ m ) be the ensemble of all m x n matrices with rows chosen 
independently and at random, with replacement, from the set of 2 n ~ k codewords of C . Furthermore, let 1 < a < [_(d — 1)/2J 
be fixed and let Q e (a,b) be the number of elementary trapping sets with parameters (a, s), < s < b, in a randomly chosen 
matrix of M.c{ m )- If 



((:)-Cr))^|[(7)^-(" 2 — ^ 



< 1, (7) 
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11 < m 

a, 6 



then P{Q e (a,b) =0} > 0. Consequently, if m satisfies (0, then there exists a parity-check matrix of C with no more than 
m + n — k — 1 rows that does not contain any (a, s) elementary trapping sets with 1 < s < b. 

Note that m + n — k — 1, with any m satisfying (|7]i represents an upper bound on the elementary trapping redundancy, i.e. 
n — k — 1. 

Proof: The proof follows the proof of Theorem 13.41 Let Ei be the event that the restriction of the z-th collection of a 
columns from a randomly chosen matrix in Mc{ m ) contains only rows of weight at most two. Among these rows, fewer than b 
rows are required to have weight one. 

Let W u denote the number of rows of weight uj in the restriction of a columns. Then 

6-1 



P i E ^ = E p i w i = 3 > (Wo + W 2 ) = (m - j)} 



3=0 
6-1 

E 

3=0 



a 2 - a + 2 



1 



2(a+l)-r. 



b-l 

E 



2 3 a J 



(8) 



where the second equation is a consequence of the fact that 

P{W x =j, (Wo + Wa) = (m-j)} = 




Equation <JHJ follows from the observation that the codewords of the dual code form an orthogonal array of strength d — 1, and 
that therefore all 1-, (")-, and Q) -collections of rows of weight 0, 1, and 2, are equally likely, respectively. ■ 
Similarly, based on the high-probability variation of LLL, one can derive upper bounds on the number of rows needed to 
guarantee that a randomly chosen matrix has no elementary trapping sets with a given set of parameters with probability at least 
1 — e. The following theorem is an analogue of Theorem 13.61 for the case of elementary trapping sets, i.e. with P{Ei] defined 
by Equation ([8]). 

Theorem 3.9: For a linear code C, let Q e (a,b) be the number of elementary trapping sets with parameters (a, s), 1 < a < 
[(d — 1)/2J, 1 < s < i < m, in an m x n array from the A4c(m) ensemble. If 



1 



2( Q H 



6-1 r 

E 

3=0 



2V • (a 2 - a + 2) r 



< 



(9) 



where r is given in Equation (O and < e < 1, then P{8 e (a. b) = 0} > 1 — e. 

For m satisfying (0 and small values of e, every randomly constructed parity-check matrix of C with no more than m + n — k—1 
rows does not contain (a, s) elementary trapping sets with 1 < s < b. 

If Ei, 1 < i < N, is used to denote the event that the i-th collection of a columns contains only rows of weight at most two, 
and less than b rows of weight one, then the result represents a straightforward application of the high-probability variation of 
LLL. The proof is therefore omitted. 



B. The Trapping Redundancy: A Constructive Approach 

The problem of finding the trapping redundancy of a linear block code can also be addressed in a deterministic manner, by 
invoking arguments similar to those used for upper bounding the stopping redundancy of codes. The results of this analysis are 
summarized in the theorem below, for the case that the parameter a of the trapping set is bounded from above by d — 1. 

Theorem 3.10: Let C be an [n, k, d] linear code. Fix the parameter a < d — 1, and let r = n — k > 2. 
Then 

TaAC) < E Q 

where b > 2 a - x - E°=t+i Q and 1 < a < 
i.e. 

T a , b (C)<J2 ( T ) +b-2 a -\ (10) 
i=i 

Furthermore, the smallest number of rows in a (redundant) parity-check matrix avoiding elementary trapping sets with parameters 
(a, s), s < 6, is upper bounded by 

T$(C)<t,( r X (ID 
Note that both claims in Theorem 13 . 1 01 also hold for all trapping sets with parameters (t, b), where t < a. 
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Proof: Let H be an arbitrary parity-check matrix of the code C of full row rank r — n — k. Consider the restriction H a 
of H on an arbitrary subset of a of its columns. Due to the fact that a < d — 1, these columns are linearly independent, so that 
the row-rank of H a must be a. Hence, there exist a rows in H a that form a basis for Fg. 

From H, form a new redundant parity-check matrix H' by adding all linear combinations of at least two, but not more than 
t < a rows of H, where t will be determined later. The total number of rows in H' in this case equals the right hand side 
of the expression in Equation ( fTOb . Since adding all possible linear combinations of not more than a rows of H to H' would 
ensure that b > 2 a ~ 1 , leaving out sums involving t + 1, a rows can reduce the number of odd-weight rows in H' a by at most 
Y^j=t+i (j)- This completes the proof of the first claim. The proof of the second claim represents a straightforward extension 
of the results in [6], and is therefore omitted. ■ 

Simple inspection reveals that the bounds in Equation ( fTOb are loose when compared to the random bounds described in the 
previous section. 

If a < d — 1 and b < r = n — k, Equation ( fTOb can be substantially tightened. In this case, it reads as 



r 



(r+1) 



T {a , b) (C)<r + 
for general trapping sets, and 

T {a , b) {C) < b ■ r 

for elementary trapping sets. 

Let I denote the number of odd-weight rows in an arbitrary restriction H a of a columns on a parity-check matrix H. Since 
H has full rank, H a contains at least one odd-weight row and therefore l<£<r — n — k holds. It is now easy to show that 
adding all linear combinations of two rows to the parity-check matrix H ensures that b > r holds for all a-sets of columns. 
For 1 < £ < r, there are £ ■ (r — £) odd-weight rows among all (?,) linear combinations of pairs of rows. Adding those linear 
combinations to the parity-check matrix brings the number of odd- weight rows to b — £ + £ ■ (r — £) — £ ■ (r — £ + 1) > r. This 
follows from the simple observation that the weight of the sum of one odd and one even weight word is always odd. Therefore, 
to avoid (a, b) trapping sets with a < d — 1, b < n — k, at most r + (V\ = r ^ 1 - > rows suffice. 

As a final remark, note that in many applications, decoding failure caused by trapping sets can only be detected upon completion 
of the decoding process. In this case, one can choose to add only a small number of judiciously chosen redundant rows to the 
parity-check matrix of the code in order to eliminate the influence of one particular trapping set. How this can be accomplished 
is illustrated on the example of the Margulis [2640, 1320] code, in Section HV-AI 



C. Asymptotic Formulas for the Trapping Redundancy 

Although there is no explicit formula for m as defined by Equations (Q~|) and (0 that holds for all possible parameter values a 
and b, such a formula can be found in the asymptotic regime (m, n — > oo, a — 0(m), b = 0(m)), by using the following results 
from [21, p. 240] and [22]. 

Let 

An = ^2 

0<i<Xm 



where < A < 1, and b — [Xm\ + 1. Since small values for the parameter b are of special interest, assume that A < 1/2. In 
this case we have 

1 



where the notation c m ~ b m describes the following relationship between two functions c m and b m of m: lirrim^oo c m /b m = 1. 
For b < m/2 + 1, with b = [Xm\ + 1, and A < 1/2, Equation (Q3 reduces to 

n\ fn-a\\ /l\ m ( m \ ^_ < i. (12) 



x ay \ a 

By invoking the well known asymptotic formula 



2J \[Xm\J 1-^ 



where H 2 ( ) denotes Shannon's binary entropy function, and ld(-) represents the logarithm with base two, it follows that m < m! 
with 



ld(e-(C)-(V)))+ld(TzV) 



1-Ho 



[Am] 



(14) 
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Also, for b < m/2 + 1, with b = [\m\ + 1, and A < 1/2, the high-probability variation of LLL given in Equation (0 reduces 

to 

Using Equation (flJl l once again results in an upper bound on the number of rows m in a parity-check matrix free of (a, s) 
trapping sets, < s < b, i.e. to < m' with 



(^) 



I-XJ \\aj \ a J ) \ V 



(16) 



Note that for sufficiently large m the right-hand side is not dependent on m as — A. 

D. Asymptotic Formulas for the Elementary Trapping Redundancy 

Likewise, it is also of interest to find an explicit formula for m given by (0 for the case of elementary trapping sets. To this 
end, we use the following asymptotic result taken from [22], stating that 

£ ( r f ) p k i rN - k - ^) ( r T) p N « rN ~ N > (17) 

k=N ^ " ' V 



where p, q > 0, p + q = 1, r > 1, N is a positive integer, and <p(y) = ^zzp- By observing that the summands in Equation (0 can 
be rewritten as 

6—1 / \ m / \ / r. 

m\ I la 



u — ± / \ fit 

E( m )2 j a' - a + 2) m ^' - (a 2 + a + 2) m £ 



it / V a 2 + a + 2 



it follows from substituting to = rN, p = ° 2 , and (7 = a 2+° +2 that 

V 2^ o* (a 2 - a + 2)"^ * ( + a + 2 ) • f m V 5 ~ 1 ^ (« 2 ~ « + 2)' 
f^ \j J W - a + 2J \m-b + l) \a 2 - a + 2/ V ; 

Using the above expression, Equation (0 reduces to 



n\ ( n — a\\ , ( a 2 + a + 2 \ / to \ / 2a \ bl /a 2 — a + 2 



v ay \ a // \ a 2 — a + 2 / \m — 6 + 1/ \ a 2 — a + 2 
This leads to a bound m < to' with 

-Id 



< 1. 



(e (C) " (V)) • <^> (S^)) " (6 - 1) ■ U 



H 2 (i^)+ld(a 2 -a + 2)-( a + l) 



(18) 

where we used Equation ( TT31 to rewrite the right hand side of the above expression. Similarly, for the high-probability variation 
of LLL and for elementary trapping sets, we obtain as a consequence of Equation (0 the bound to < to' with 



(0 - (Y) - 1) ■ ld (jfy (1 - t) ) ) ~ ld (&SH)) - (6 - 1) • id 

H 2 (i^I) + ld(a 2 -a + 2)-(a+l) 



2a 



a 2 -o+2 



(19) 



Based on the results of the previous sections, it is also straightforward to see that the asymptotic formula for the trapping 
redundancy of Theorem 13.101 is of the form 

T a ,b(C)< (, r ) —^ r +b-2^~\ (20) 
\l ar \J 1- — 

where a = [ar\. 

Remark: Note that the matrices from the ensemble Aic(m), for large to, may have highly non-uniform row and column 
weights. The variable- and check-node degrees of their corresponding Tanner graphs may be very large, leading to the emergence 
of short cycles. It is therefore important to compare the derived bounds with some benchmark values, the latter corresponding to 
redundant matrices that are known to have a small number of redundant rows, no short cycles, as well as no small trapping sets. 
Two such examples, including the aforementioned Margulis and projective geometry codes, are discussed in the next section. Other 
families of codes, such as codes based on Latin squares and designs, are analyzed in more detail in the companion paper [23]. 
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IV. Trapping Redundancy: Analytical Comparisons 

We perform next a numerical study of the probabilistic upper bounds derived in Section [HI] for the Margulis [2640, 1320] code 
and the class of projective geometry codes. The goal of the comparative study is to both assess the tightness of the bounds of 
Section [Til] and to demonstrate that structured LDPC codes with redundant parity-check matrices can avoid small trapping sets in 
their Tanner graphs. 

Note that the presented results only capture the trade-off between the smallest size of general and elementary trapping sets 
and the number of rows in the corresponding (redundant) parity-check matrix, without taking into consideration other important 
matrix properties such as variable and check nodes degree, girth, and cycle length distribution. 

A. The [2640, 1320] Margulis Code 

Numerical values of the trapping redundancy derived in Section [111] for the [2640, 1320] Margulis code are listed in Tables U 
and |nl The labels LLL (std), LLL (hp) refer to the bounds based on LLL in standard form and its high-probability variation, 
respectively. The symbol m denotes the number of rows required by the LLL approach, while rh refers to the number of rows 
of a redundant, rank n — k parity-check matrix, which is an upper bound on the trapping redundancy, T Q) t>(C) < rh. As the exact 
minimum distance is not known for this code, a method [24] for approximating the minimum distance, proposed in [25], was 
used instead. The estimate at hand is d « 40, and we restrict our attention to values of the parameter a strictly (and significantly) 
smaller than a/2 = 20. 

The full-rank 1320 x 2640 parity-check matrix H of the Margulis code, constructed in the standard manner [26], contains 
no cycles of length less than eight, but includes a large number of (12,4) and (14,4) elementary trapping sets [3], [16]. The 
LLL-based bounds reveal that there exists a matrix of full rank with at most 1336 rows that does not contain (14, s) elementary 
trapping sets, with s < 5, and that adding at most 19 additional rows ensures that the matrix does not contain (12,4) trapping 
sets either. 

It can also be seen from Table U that there exists a matrix of row -rank n — k with rh = 1394 rows that is free of trapping sets 
of size (6, s), s < 5. However, for a parity-check matrix free of elementary trapping sets of the same parameters, Table ITTI shows 
that only rh — 1351 rows are needed. 

These bounds cover the case of fixed a values only. Note that finding analogs of these results that cover a range of general 
trapping set sizes a instead may be desirable for certain practical applications. Due to the monotonic increase of the trapping 
redundancy with the value of the parameter a, one can see that if a set of rows, randomly drawn from the codewords of the dual 
code, does not contain trapping sets of size a with high probability, then this set is also very unlikely to support trapping sets of 
size smaller than a. For example, to obtain a matrix free of (14, s) trapping sets, s < 5, according to the high-probability version 
of LLL with e = 10~ 20 one needs m — 216 rows, so that rh = 1535. With high probability, this matrix does also avoid (12, s) 
trapping sets, s < 5, due to the LLL-based study. 

Note that for elementary trapping sets, Table U indicates that the larger the value of the parameter a, the smaller the number 
of redundant rows that is needed to eliminate such trapping sets. This result may seem counterintuitive, but it follows from the 
fact that trapping sets are deemed elementary only as their restriction does not contain rows of weight larger than two - an event 
that becomes less likely with the increase of the parameter a. 
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5 


1320 x 2640 


75 


1394 


87 
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150 1469 
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1320 x 2640 
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167 1486 


12 
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1320 x 2640 
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1448 


138 


1457 


200 1519 


14 
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1320 x 2640 


145 


1464 


154 
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216 1535 



TABLE I 

Upper bounds on the (a, b) trapping redundancy T a ^ b (C) of the Margulis code. 
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1348 
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12 


5 


1320 x 2640 
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1339 


31 1350 


14 
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1320 x 2640 


17 


1336 


18 


1337 


26 1345 



TABLE II 

Upper bounds on the (a, b) elementary trapping redundancy (C) of the Margulis code. 
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Fig. 1. Trapping set structure of a (12, 4) trapping set and its expansion, (a) first configuration; (b) second configuration. 



We complete this section by illustrating how a simple structured method can be used to add only one redundant parity-check 
equation so as to increase the value of the parameter b in any given (a =12,6 = 4) or (a = 14, b = 4) trapping set. 

Example 4.1: The knowledge about the structure of specific trapping sets - such as the (12,4) and (14,4) trapping sets - in 
the [2640, 1320] Margulis code can be used to eliminate single instances of such sets. 

First, we observe that the support of any (14,4) trapping set contains the support of a (12,4) trapping set - i.e., the sets 
are nested. Throughout the remainder of this section, we call the variables and checks introduced by extending a (12,4) to a 
(14, 4) trapping set expansion variables and expansion checks, respectively. The notation B, E, and O is used to refer to the basic 
(12,4) trapping set, its expansion variables and checks, and the graph outside (i.e. complementary to) the (14,4) trapping set, 
respectively. Since the Margulis code is regular, with variable node degree d v = 3 and check node degree d c — 6, an elementary 
(a. b) trapping set with a variables and b check nodes of degree on^| has a fixed number of checks. Therefore, in order to extend 
an elementary (12,4) trapping set to a (14,4) trapping set, two variables and three checks have to be added. The notions of 
basic, expansion and outside variables and check nodes are illustrated in Figure [TJ 

Since the Margulis code has no four-cycles, at most one expansion check can be connected to both the expansion variables. 
Consequently, either three or four edges emanating from the expansion variables are connected to expansion check nodes, while 
the remaining edges are connected to check nodes whose degree in the basic (12,4) trapping set is one. Thus there exist only 
two possible configurations for such trapping sets, as shown in Figure [TJ 

Now, based only on the knowledge of check nodes of degree one within the basic (12,4) trapping set, it is straightforward to 
determine the whole expansion set: in the first case, the two degree-one checks of the basic (12,4) trapping set are connected 
through two variable nodes to one additional check node. These two variable nodes are the expansion variables. In the second 
configuration, the expansion variables do not share a check node. 

For the first configuration, denote the two expansion variables by i>e,i and i/e,2- Furthermore, denote the check node connected 
to both these variables by ce- The degree-one check nodes in the basic trapping set neighboring the expansion variables are 
denoted by cbe.i and cbe,2, while the variable nodes in the basic trapping set connected to check nodes cbe.i and Cbe,2 are 
denoted by i>be,i and i>be,2, respectively. The check nodes of degree one in the expansion of the trapping set are named ceo,i and 
ceo, 2- The two remaining check nodes of degree one in the basic trapping set are termed cbo,i and cbo,2, and the variable nodes 
within the basic (12,4) trapping set connected to them are wbo,i and vbo,2, respectively (see Figure [TJ a)). The configuration 
involving all the aforementioned checks and variables is illustrated in Table [TTTT 
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1 





TABLE III 



Restriction of the basic trapping set and the expansion variables 



These check nodes, incidentally, correspond to unsatisfied check nodes, that can be readily identified upon termination of the iterative decoding process. 
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The parity-check equations containing the restriction described in Table [TTT] can be linearly combined to generate a redundant 
parity-check equation that has a restriction of odd weight in both the (12, 4) and the (14, 4) trapping set. There are three different 
methods for linearly combining the parity-check equations with restrictions as shown in Table Hill 

Method Si refers to adding the rows indexed by (c E , c E o,i> c B e,2), (ce, c E o,2, c B e,i), (ceo,i, c B e,iX and (c E o,2, c BEj2 )- 
Method 5*2 refers to adding the rows indexed by (ce, ceo.i) and ( c e, c B o,2)- Observe that all these combinations have a restriction 
of weight one within the expansion variables. Method S3 differs from the previous methods in so far that it may generate redundant 
parity-check equations with odd-weight restrictions that are not necessarily of weight one. Candidate equations are obtained by 
adding the rows indexed by (c E , c BO ,i), (c E , c BO ,2), (ce, Cbe,i, c be , 2 , c bo ,i), (c e , Cbe,i, c be ,2, c bo ,2), (cbo,i> c B o,2, c EO ,i, 
c B e,2), and (c B o.i, c B o,2, ceo.2, cbe.i)- The Hamming weight of the constructed rows is an even integer between 10 and 24. 
The lower bound 10 is obtained if the supports of two added parity-checks share exactly one element. The intersection of the 
supports cannot have more than one element, since the code has no four cycles. The upper bound 24 is met when four parity-check 
equations are added and none of the variables listed in Table [Til] occur in the support of more than one parity-check. 

If a trapping set of the form shown in Figure [Tib) is present in the code graph, then the two expansion variables cannot have 
a common check node. Consequently, the expansion variable ve,i is connected to two of the degree-one check nodes of the 
basic trapping set0 If there is only one variable node connected to two degree-one checks, denoted by c B e,i and c B e,2. the 
variable of interest is the expansion variable t> E 1, which is also connected to expansion check node c E o,i- Observe that the 
expansion variable vb,2 is strongly influenced by its two neighboring check nodes connected to the outside graph and it cannot 
be uniquely determined. Due to this limited knowledge of the expansion set, there are only two possible ways to generate a 
redundant parity-check with an odd restriction weight on the basic trapping set, involving the sums of the rows (c E o,i> cbe.i) as 
well as (c E o,i, c B e,2). A similar analysis can be conducted for (14, 4) trapping sets. Details regarding this procedure are omitted. 

As illustrated by the example, knowledge about the structure of trapping sets allows one to exactly determine the choice of 
the redundant row to be added to the parity check matrix. Unfortunately, since there are at least 1320 trapping sets of each such 
form in the code graph, adding this many rows to the code matrix is undesirable. Nevertheless, as already pointed out, only one 
row can be added upon detecting the presence of a decoding failure caused by a given trapping set. 

B. Projective Geometry Codes 

Projective geometry codes are linear block codes with many well known combinatorial parameters and properties. As will be 
shown next, it is also straightforward to characterize a large sub-family of trapping sets in these codes. 
We start our derivations by introducing the relevant terminology. 

Definition 4.1: [27] A finite projective geometry PG(M, q) of dimension M, over a finite field GF(g), for some prime power 
q, is a set of points and subsets thereof, called lines. The following axioms hold for the points and lines of a finite geometry: 

• Two distinct points determine a unique line. 

• Every line consists of more than two points. 

• For every pair of distinct lines L\ and L2, intersecting at some point r, there exist two pairs of points (pi,gi) G L\ and 
{P2, 92) € L2 that differ from r, such that the lines determined by (pi,p2) and (gi, q%) intersect as well. 

• For each point and for each line, there exist at least two lines and two points that are not incident to them, respectively. 
The points of a projective geometry PG(M, g) can be represented by non-zero (M + l)-tuples (ao, a\, 0,2, clm) such that 
a, G GF(g). Points of the form (ao, ai, Q>2, &m) and (Sao, Sai, 5a,2, Sqm)> 8 G GF(g)\{0}, are considered equivalent. 
A line through two distinct points (ao, cii, 02, ••■! o>m) and (po, b\, 62, &m) consists of all points that can be expressed as 
(a ao + (3 bo, a am + (3 6a/), where a, (3 G GF(g) and are not both simultaneously zero. Consequently, a projective geometry 
PG(M, q) has (q M+1 — l)/(q— 1) points, and each line in the geometry contains g+1 points. The number of lines in a projective 
geometry is given by 

(g M + ... + g + l)(g M - 1 + ... + g + l)/(g + l). (21) 

It is straightforward to see that the number of lines and points coincide for M = 2, since (q 2 + q + 1) = (g 3 — l)/(g — 1) 
holds. 

A type-I projective geometry code is defined in terms of a parity-check matrix representing the line-point incidence matrix of 
a projective geometry PG(M, g) [28]. Throughout the remainder of the paper, we consider projective plane codes, M = 2, and 
codes based on projective geometries with M = 3 only. 

Definition 4.2: An s-arc in PG(2, g) is a collection of s points such that no three of them are collinear. The lines incident to 
an s-arc JC are either unisecants (they intersect the arc in exactly one point) or bisecants (they intersect the arc in exactly two 
points). Similarly, an s-cap in PG(3,g) is a set of s points, no three of which are collinear. 
The following results pertaining to unisecants and bisecants are taken from [29, Ch. 8] and [30, Ch. 16]. 

Lemma 4.3: Let n\ and n-x denote the number of unisecants and bisecants of an s-arc K. in PG(2, q), respectively. Then 

m = s(g + 2 - s), and ri2 = -s(s-l). (22) 

3 One must keep in mind that the choice for such a variable may not be unique. 
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Similarly, for an s-cap JC in PG(3, q) it holds that 

Hi = s(q 2 + q + 2 - s), 



(23) 



where rt\ denotes the number of unisecants of K. 

Lemma 4.4: The largest arc in PG(2, q) contains at most q + 2 points, for q even, and q + 1 points, for q odd. Arcs with 
s = q + 1 and s = q + 2 are called ovals and hyperovals, respectively. The size of any s-cap in PG(3, q) satisfies s < q 2 + 1. 
For q > 2, a (q 2 + l)-cap is called an ovaloid. 

The results of Lemmas 14.31 and 14.41 can be used to establish the following simple results regarding trapping sets in the Tanner 
graph of type-I projective geometry codes. Note that all stated results restrict the parameter sets for which trapping sets may 
potentially exist, although they do not imply the existence of such sets. 

Corollary 4.5: All elementary trapping sets of a PG(2, q), type-I, projective geometry code have parameters (s, s(q + 2 — s)). 
Consequently, the number of degree-one check nodes of such trapping sets for q odd is necessarily larger than or equal to the 
number of variables in the trapping set. For even values of q, an exception to the aforementioned rule is a hyperoval, which 
represents a codeword. The trapping sets with the smallest ratio b/a have parameters (q + 1, q + 1) (q odd) and (q + 2, 0) (q 
even), respectively, and those with the largest ratio (3, 3(q — 1)). 

Proof: Note that the parity-check matrix if of a PG(2, q) code is the line-point incidence matrix of the underlying PG. 
Arcs correspond to a collection of columns, the restriction of which has rows of weight at most two only, and exactly ni of these 
rows have weight one. This is equivalent to the definition of an (s,ni) trapping set, where s > 3 and also s < (q+ 1) (q odd) 
or s < (q + 2) (q even), respectively, according to Lemma l4~4l The smallest and largest ratios of b/a are defined by the limits of 
s. Hyperovals have s = q + 2 points and n\ = (q + 2) • (q + 2 — (q + 2)) = unisecants, and therefore correspond to (q + 2, 0) 
trapping sets, which are codewords. ■ 

Corollary 4.6: All elementary trapping sets of a PG(3, q), type-I projective geometry code have parameters (s, s(q 2 +q+2 — s)). 
Provided that the PG contains an s-arc with ri\ = s(q 2 + q + 2 — s), the trapping sets with the smallest and largest ratio b/a 
have parameters (q 2 + 1, (q 2 + l)(q + 1)) and (3, 3(q 2 + q — 1)), respectively. 

Proof: The proof follows along the lines of the proof of Corollary 14.51 with n\ = s(q 2 + q + 2 — s) for PG(3, q). ■ 

A complete classification of trapping sets in projective geometry codes is probably an impossible task. This is due to the fact 
that very little is known about the number and existence of arcs and caps of different sizes in projective spaces. One aspect of this 
problem that is better understood is the existence and enumeration of complete arcs (and caps) - i.e. arcs and caps not contained 
in any larger arc or cap. The interested reader is referred to [29], [30] for more information regarding the problem of complete 
arc enumeration. 

We compare next the upper bounds derived in Section [ill] with the results of the study presented in this section. We consider 
elementary trapping sets only. 

In order to apply the results of the lemmas in this section, one has to consider a parity-check matrix that represents a complete 
line-point incidence structure [28]. For this reason, the standard parity-check matrices of PG codes contain exactly n rows, and 
are therefore redundant. 

Table [IV] list the number of rows required to avoid elementary trapping sets of a given size, computed according to LLL and 
its high-probability variation. The values for to are derived using the minimum distance and the rank results taken from [28]. 

Observe that Table ITVl indicates that there exists a parity-check matrix for the PG(2, 16) code with at most 175 rows and no 
(3, s), s < 45, elementary trapping sets. This is significantly less than n — 273 as for the PG code, but might also include rows 
of large weight. On the other hand, to obtain a matrix without such trapping sets with probability larger than 1 — 10 -20 , at most 
309 rows are required, a number clearly larger than n. 
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TABLE IV 

Upper bounds on the (a, 6) elementary trapping redundancy of PG codes. 



Although there exist parity-check matrices for the PG codes that have smaller row-redundancy and do not contain elementary 
trapping sets of small sizes, the matrix defined by the PG construction presents a way to generate a parity-check matrix with 
limited redundancy, small row-weight, and therefore few short cycles, which also performs well in the waterfall region. As can be 
seen from the comparison table, PG codes represent an attractive systematic construction of LDPC codes without small trapping 
sets, whose redundancy lies between the bounds based on the standard case and the high-probability version of LLL. 
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V. Conclusions 

We introduced the notion of the (a, b) trapping redundancy of a code, representing the smallest number of rows in any parity- 
check matrix of the code that avoids (a, s) trapping sets with 1 < s < b. Upper bounds on these combinatorial numbers were 
derived using Lovasz Local Lemma and variations thereof. Also presented were numerical results for the trapping redundancy of 
the Margulis [2640, 1320] and type-I PG codes. 

Acknowledgment: The authors are grateful to the anonymous reviewers for their constructive comments that significantly 
improved the exposition of the work. In addition, they would like to thank Dr. Richardson for carefully handling the manuscript. 
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